Bayesian Approach to Parameter Estimation

In many problems, before any observations from f(x|θ) are available, the experimenter or statistician will be able to summarize his previous information and knowledge about where in Θ the value of θ is likely to lie by constructing a probability distribution for θ on the set Θ. In other words, before any experimental data have been collected or observed, the experimenter’s past experience and knowledge will lead him to believe that θ is more likely to lie in certain regions of Θ than in others. We would assume that the relative likelihoods of the different regions can be expressed in terms of a probability distribution on Θ. This distribution is called the prior distribution of θ, because it represents the relative likelihood that the true value of θ lies in each of various regions of Θ prior to observing any values from f(x|θ).